Layer decomposition in hierarchical VDR coding

ABSTRACT

Techniques use multiple lower bit depth (e.g., 8 bits) codecs to provide higher bit depth (e.g., 12+ bits) high dynamic range images from an upstream device to a downstream device. Multiple layers comprising a base layer and one or more enhancement layers may be used to carry video signals comprising image data compressed by lower bit depth encoders to a downstream device, wherein the base layer cannot be decoded and viewed on its own. Lower bit depth input image data to base layer processing may be generated from higher bit depth high dynamic range input image data via advanced quantization to minimize the volume of image data to be carried by enhancement layer video signals. The image data in the enhancement layer video signals may comprise residual values, quantization parameters, and mapping parameters based in part on a prediction method corresponding to a specific method used in the advanced quantization. Adaptive dynamic range adaptation techniques take into consideration special transition effects, such as fade-in and fade-outs, for improved coding performance.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/351,647, filed on Apr. 14, 2014, which is the US national stageapplication for PCT Application Ser. No. PCT/US2012/062932, which claimspriority to U.S. Provisional Patent Application No. 61/555,978 filed onNov. 4, 2011, and U.S. Provisional Application No. 61/596,600 filed onFeb. 8, 2012, all of which are hereby incorporated by reference in theirentirety.

TECHNOLOGY

The present invention relates generally to image processing, and inparticular, to encoding, decoding, and representing variable dynamicrange images using a hierarchical VDR codec.

BACKGROUND

Display technologies being developed by Dolby Laboratories, Inc., andothers, are able to reproduce images having high dynamic range (HDR).Such displays can reproduce images that more faithfully representreal-world scenes than conventional displays.

To support backwards compatibility as well as new HDR displaytechnologies, multiple layers may be used to deliver video data from anupstream device such as a multi-layer video encoder to downstreamdevices. Standard dynamic range (SDR) video data carried in a base layer(BL) of the multiple layers is optimized for viewing experience on SDRdisplays, while visual dynamic range (VDR) video data carried in thecombination of the base layer and an enhancement layer (EL) of themultiple layers supports viewing experience of VDR displays havinghigher dynamic ranges than that of SDR displays. As used herein, codecsinvolved in encoding and decoding such image data are denoted as VDRcodecs optimized for SDR displays.

BL image data may comprise lower bit depth (e.g., 8 bits per colorcomponent) SDR images derived from higher bit depth (e.g., 12 or morebits per color component) HDR source images from an image data input.The SDR images encoded in the BL image data typically comprisecolor-corrections by colorists to make the SDR images look as realisticas possible within a relatively narrow or standard dynamic range. Forexample, hue information related to some or all of the pixels in aninput HDR image may be changed or corrected in an SDR image in order tocreate a realistic looking image within the standard dynamic range.These color corrections result in asymmetric clippings in various colorchannels, and introduce manual color alterations especially inrelatively underexposed or overexposed regions of the HDR source images.The color corrected SDR image may allow SDR displays to show imagedetails in the dark areas and highlights of an HDR source image.

Clipping is a type of color alternation that alters/modifiesout-of-bound pixel values in color channels so that the resultant pixelvalues are within a target represented range (which may be one within arange supported by a specific type of SDR displays, or within a rangesupported by a range of SDR displays, or within a range supported by arange of VDR displays, etc.). Clipping may occur in zero, one or more ofcolor channels (e.g., R, G, and B pixel values in a RGB color space in acertain portion of a HDR image may be clipped in a tone-mapped image).Amounts of clipping may or may not vary with the color channels (e.g.,more clipping for green, less clipping for blue, etc.).

Color corrections, such as clipping, introduced into SDR images make theSDR images to comprise different and independently sourced imagecontents from their counterpart VDR images, and are difficult and evenimpossible to remove by a downstream device for the purpose ofreconstructing high dynamic range images without complicated processingand without a sufficiently large bitrate. When multiple layers are usedto transmit image data to a downstream device, reversing colorcorrections may require a large volume of additional image data to betransmitted, for example, in an enhancement layer, to the downstreamdevice.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection. Similarly, issues identified with respect to one or moreapproaches should not assume to have been recognized in any prior art onthe basis of this section, unless otherwise indicated.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1A and FIG. 1B (FIG. 1) illustrate a visual dynamic range codecarchitecture in a baseline profile, in accordance with an exampleembodiment;

FIG. 2A and FIG. 2B (FIG. 2) illustrates a visual dynamic range codecarchitecture in a main profile, in accordance with an exampleembodiment;

FIG. 3 illustrates scene-adaptive dynamic range adjustment quantizationas applied in an YCbCr color space, in accordance with an exampleembodiment;

FIG. 4A and FIG. 4B illustrate example process flows, according toexample embodiments of the present invention;

FIG. 5 illustrates an example hardware platform on which a computer or acomputing device as described herein may be implemented, according anembodiment of the present invention; and

FIG. 6 illustrates an example flow for detecting transition sequencesand selecting among two quantization schemes, according to an embodimentof the present invention.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Example embodiments, which relate to encoding, decoding, andrepresenting variable dynamic range images using a hierarchical VDRcodec, are described herein. In the following description, for thepurposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent, however, that the present invention may be practicedwithout these specific details. In other instances, well-knownstructures and devices are not described in exhaustive detail, in orderto avoid unnecessarily occluding, obscuring, or obfuscating the presentinvention.

Example embodiments are described herein according to the followingoutline:

-   -   1. GENERAL OVERVIEW    -   2. HIERARCHICAL VIDEO DELIVERY        -   2.1. BASELINE PROFILE        -   2.2. MAIN PROFILE    -   3. ADVANCED QUANTIZATION    -   4. LINEAR STRETCHING    -   5. EXAMPLE PROCESS FLOWS    -   6. ADAPTIVE DYNAMIC RANGE ADJUSTMENT    -   7. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW    -   8. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

1. GENERAL OVERVIEW

This overview presents a basic description of some aspects of an exampleembodiment of the present invention. It should be noted that thisoverview is not an extensive or exhaustive summary of aspects of theexample embodiment. Moreover, it should be noted that this overview isnot intended to be understood as identifying any particularlysignificant aspects or elements of the example embodiment, nor asdelineating any scope of the example embodiment in particular, nor theinvention in general. This overview merely presents some concepts thatrelate to the example embodiment in a condensed and simplified format,and should be understood as merely a conceptual prelude to a moredetailed description of example embodiments that follows below.

In some embodiments, hierarchical VDR codecs may be used to providecompressed VDR images (e.g., video images) to VDR image processingdevices (e.g., VDR displays). As used herein, the term “hierarchical VDRcodec” may refer to a VDR codec wherein the base layer may not be viewedby itself on SDR displays. As used herein, the term “VDR” or “visualdynamic range” may refer to a dynamic range wider than a standarddynamic range, and may include, but is not limited to, a wide dynamicrange up to the instantaneously perceivable dynamic range and colorgamut which human vision can perceive at an instant.

A hierarchical VDR codec as described herein that supports higher bitdepth (e.g., 12+ bits) VDR images may be implemented with two or morelower bit depth (e.g., 8 bits) codecs in multiple layers. The multiplelayers may comprise a base layer and one or more enhancement layers.

In sharp contrast with other techniques, base layer image data undertechniques as described herein is not to support optimized viewing onSDR displays, or to make SDR images to look as good as possible,matching human perceptual within a standard dynamic range. Instead, thebase layer image data under techniques as described herein is to supportoptimized viewing on VDR displays. In an example embodiment, the baselayer image data under techniques as described herein comprises aspecific constitution of a lower-bit depth version of VDR image data andthe remaining difference between base layer and the original VDR imageis carried in the enhancement layer.

Also, under other techniques, VDR image data and SDR image data relatingto the same source images comprise different image contents. Forexample, input SDR image data to an encoder comprises ad hoc independentalterations that are not known or determinable from input VDR image datato the encoder. Oftentimes, color corrections or results of colorgrading by a colorist must be forensically analyzed by comparing the SDRimage data with the VDR image after the SDR image data has already beenaltered, for example, by the colorist.

In sharp contrast, under techniques as described herein, VDR image datamay be used to derive base layer (BL) image data via hierarchicaldecomposition, e.g., advanced quantization followed by layered coding.Specific methods applied in the advanced quantization are known and evenselected purposefully by a hierarchical VDR encoder. Theselection/determination of a particular advanced quantizer to performadvanced quantization may be based on, for example, how the imagequality of reconstructed VDR images may be on the VDR decoder side.Hence, advanced quantization under techniques as described herein is oneor more operations known a priori (e.g., before the input uncompressedbase layer data to base layer processing is produced), controlled, andimplemented by a hierarchical VDR codec as described herein. Thus,complex analysis to determine differences between the VDR image data andthe SDR image data which is independently altered or generated underother techniques can be avoided or disabled under techniques asdescribed herein.

Codecs implementing techniques as described herein may be configured toinclude inter-layer prediction capabilities to fully exploit statisticalredundancy between base layer (BL) image data and original input VDRimage data. EL image data may (possibly only) carry residual (ordifferential) image data, instead of carrying a large amount of VDRimage data without exploiting the statistical redundancy in image dataof different layers.

In some embodiments, prediction may be used to further minimize theamount of VDR image data that is to be carried in the enhancementlayers. As a specific application of advanced hierarchical VDR encoder,a corresponding relationship may be established by the hierarchical VDRencoder between advanced quantization and prediction. Based on thespecific application of advanced quantization used to derive the inputuncompressed base layer data to base layer processing, the hierarchicalVDR encoder may select a specific corresponding prediction method amonga plurality of available prediction methods. In an example, if linearquantization is used in the advanced quantization, a first orderpolynomial based prediction method may be used for prediction. Inanother example, if a quantization curve (e.g., Sigmoid curve, mu-law, ahuman-perceptual based curve, etc.) is used in the advancedquantization, a higher-order (second order or higher) polynomial basedprediction method that corresponds to the quantization curve may be usedfor prediction. In another example, if a cross-color (vector) channelquantization (e.g., slope/offset/power/hue/saturation used in primarycolor grading operation) is used in the advanced quantization, acorresponding cross-color channel prediction may be used for prediction.In yet another example, if a piecewise quantization is used in theadvanced quantization, a prediction method corresponding to thepiecewise quantization may be used for prediction. A correspondingprediction method may be preconfigured or dynamically selected by thehierarchical VDR encoder, since the hierarchical VDR encoder knows inadvance (e.g., without analyzing the result of the advancedquantization) whether, and which specific type of, e.g., a linearquantization, a curved quantization, a cross-color channel quantization,a piecewise quantization, a look up table (LUT) based quantization, aspecific combination of different types of quantizations, etc., is usedin the advanced quantization.

In sharp contrast, under other techniques, as color corrections to theinput SDR image data in a base layer, such as those made by a colorist,are independently performed, it is difficult to determine which methodshould be applied for prediction without expensive comparison andanalysis processing over independently differing image contents of boththe input SDR image data in the base layer and the input VDR image data.

Thus, in some embodiments, complex and expensive analysis (e.g., inprediction operations) to determine differences in VDR and independentlyaltered input base layer contents may be disabled or avoided undertechniques as described herein. A hierarchical VDR codec may implementadvanced quantization and processing logic to correlate the advancedquantization with prediction.

In some embodiments, even though a hierarchical VDR codec is notdesigned to provide base layer image data optimized for viewing in SDRdisplays, the hierarchical VDR codec may still extensively reusecomponents in a VDR codec with base layer optimization. In anembodiment, a hierarchical VDR encoder may add one or more modules to,or modify one or more modules in, a VDR codec infrastructure optimizedfor SDR displays to generate input base layer image via advancedquantization to base layer processing from input VDR image data. Thus,the hierarchical VDR encoder may need only a single input of imagecontent from the input VDR image data rather than one input of imagecontent for VDR and another input of differing image content for SDR.For example, a conversion module in the hierarchical VDR encoder mayimplement advanced quantization to convert input 16 bit RGB VDR data to8 bit YCbCr as input base layer image data to base layer processing.

In an example embodiment, a hierarchical VDR codec may be configured toextensively support the VDR reference processing syntax, specification,and coding architecture, as defined, for example, in an industrystandard, a proprietary specification, an extension from an industrystandard, or a combination of the foregoing. In an example embodiment,one or more of inputs and outputs of the hierarchical VDR codec (encoderand/or decoder) are the same as, or substantially similar to, thosespecified by the VDR specification or profiles for a VDR codec optimizedfor SDR displays. A hierarchical VDR codec may be a vehicle to processand render 12+ bits VDR images via two (inexpensive) 8 bit decoders,obviating a need to use an expensive 12+ bit decoder to provideperceptually similar image quality for VDR images. As used herein, theterm “N+ bit image” may refer to images that are represented using Nbits or more per color component and have at least one color component.In some embodiments, more than one lower bit depth decoder in a codecand/or more than one lower bit depth encoder may work in parallel atleast for some operations and jointly perform encoding and decoding ofVDR image data in a device.

Practical benefits of the embodiments described herein include, but arenot limited only to, providing high quality VDR image data to endconsumers who only care about the final VDR quality and do not care oreven look at the SDR version that might be constructed from base layerimage data.

In some embodiments, a combined codec (which may be a VDR encoder or aVDR decoder) may be used to operate in multiple modes. One of theoperational modes for the combined codec may place the combined codec tooperate as a hierarchical VDR codec, whereas a different one of theoperational modes for the combined codec may also allow for encoding abase layer that is suitable to be viewed on SDR displays. As a result,in some example embodiments, coded bitstreams that comply with either ofthe VDR specifications may be properly decoded by the combined VDRdecoder. As a result, in some example embodiments, coded bitstreams thatcomply with either of the VDR specifications may be properly generatedby the combined VDR encoder.

In some example embodiments, data needed for other applications may alsobe included with base layer and enhancement layer image data to bedelivered from an upstream device to a downstream device. In someexample embodiments, additional features and/or orthogonal features maybe supported by the base and enhancement layers as described herein.

In some example embodiments, mechanisms as described herein form a partof a media processing system, including but not limited to any of: ahandheld device, game machine, television, laptop computer, netbookcomputer, tablet computer, cellular radiotelephone, electronic bookreader, point of sale terminal, desktop computer, computer workstation,computer kiosk, or various other kinds of terminals and media processingunits.

Various modifications to the preferred embodiments and the genericprinciples and features described herein will be readily apparent tothose skilled in the art. Thus, the disclosure is not intended to belimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features described herein.

2. HIERARCHICAL VIDEO DELIVERY

In some embodiments, a base layer and one or more enhancement layers maybe used, for example by an upstream device (e.g., a VDR image encoder102 of FIG. 1A or a VDR image encoder 202 of FIG. 2A), to deliver imagedata in one or more video signals (or coded bitstreams) to a downstreamdevice (e.g., a VDR image decoder 150 of FIG. 1B). The image data maycomprise base layer image data of a lower bit depth quantized from ahigher bit depth (e.g., 12+ bits) VDR image and carried in a base layerimage container (a YCbCr 4:2:0 image container), and enhancement layerimage data comprising residual values between the VDR image and aprediction frame generated from the base layer image data. The baselayer image data and the enhancement layer image data may be receivedand used by the downstream device to reconstruct a higher bit depth (12+bits) version of the VDR image.

In some embodiments, the base layer image data is not for producing aSDR image optimized for viewing on SDR displays; instead, the base layerimage data, together with the enhancement layer image data, is optimizedfor reconstructing high quality VDR images for viewing on VDR displays.

2.1. Baseline Profile

FIG. 1A and FIG. 1B show a VDR codec architecture in a baseline profile,in accordance with an example embodiment. As used herein, the termbaseline profile may refer to the simplest encoder profile in a VDRcoding system. In an embodiment, baseline profile restricts all videoprocessing in the base and enhancement coding layers in the YCbCr 4:2:0color space. In an example embodiment, prediction may be made with anYCbCr space under a 4:2:0 sampling scheme; a polynomial/1D LUTprediction method, for example, may be used for prediction. In someembodiments, an upstream device that delivers VDR image data todownstream devices may comprise a VDR image encoder 102 implementing oneor more techniques as described herein, while a downstream device thatreceives and processes video signals from the VDR image encoder 102 maycomprise a VDR image decoder 150 implementing one or more techniques asdescribed herein. Each of the VDR image encoder 102 and the VDR imagedecoder 150 may be implemented by one or more computing devices.

In an example embodiment, the VDR image encoder (102) is configured toreceive an input VDR image (106). As used herein, an “input VDR image”refers to wide or high dynamic range image data that may be used toderive a VDR version of a source image (e.g., raw image captured by ahigh-end image acquisition device, etc.), which gives rise to the inputVDR image. The input VDR image may be in any color space that supports ahigh dynamic range color gamut. In some embodiments, the input VDR image(106) is the only input, relative to the source image, that providesimage data for the VDR image encoder (102) to encode; input image data,relative to the source image, for base layer processing under techniquesas described herein may be generated based on the input VDR image (106)using advanced quantization.

In an example embodiment, the input VDR image is a 12+ bit RGB image inan RGB color space, as illustrated in FIG. 1A. In an example, each pixelrepresented in the input VDR image comprises pixel values for allchannels (e.g., red, green, and blue color channels) defined for a colorspace (e.g., a RGB color space). Each pixel may optionally and/oralternatively comprise upsampled or downsampled pixel values for one ormore of the channels in the color space. It should be noted that in someembodiments, in addition to three primary colors such as red, green andblue, different primary colors may be concurrently used in a color spaceas described herein, for example, to support a wide color gamut; inthose embodiments, image data as described herein includes additionalpixel values for those different primary colors and may be concurrentlyprocessed by techniques as described herein.

In an example embodiment, the VDR image encoder (102) is configured totransform pixel values of the input VDR image from a first color space(e.g., a RGB color space) to a second color space (e.g., an YCbCr colorspace). The color space transformation may be performed, for example, bya RGB-2-YCbCr unit (108) in the VDR image encoder (102).

In an example embodiment, the VDR image encoder (102), or a downsampler(e.g., a 444-420 downsampler 110) therein, is configured to downsamplethe VDR image (e.g., in a 4:4:4 sampling format) in the YCbCr colorspace into a 12+ bit downsampled VDR image 112 (e.g., in a 4:2:0sampling format). Without considering the effects of compression, thetotal amount of image data in a chroma channel of the 12 bit+downsampled VDR image (112) may be one quarter in size of the totalamount of image data in a luminance channel of the 12 bit+ downsampledVDR image (112).

In an example embodiment, the VDR image encoder (102) is configured toperform advanced quantization on YCbCr image data (in the 4:2:0 samplingformat in the present example), as downsampled from the VDR image (inthe 4:4:4 sampling format), to generate an 8 bit BL image (114) in theYCbCr color space. As illustrated in FIG. 1A, both the 12+ bit VDR image(112) and the 8 bit BL image (114) are generated after the same chromadownsampling and hence contain the same image content (e.g., the 8 bitBL image 114 being more coarsely quantized than the 12+ bit VDR image112).

In an example embodiment, the VDR image encoder (102), or a firstencoder (116-1) therein, is configured to encode/format the 8 bit BLimage (214) in the YCbCr color space into image data into a base layerimage container in the 4:2:0 sampling format. In some embodiments, theimage data in the base layer image container is not for producing SDRimages optimized for viewing on SDR displays; rather, the image data inthe base layer image container is optimized to contain an optimal amountof base layer image data in a lower bit depth image container for thepurpose of minimizing an overall bit requirement for VDR image data tobe carried in multiple layer to be reconstructed into a VDR imageoptimized for VDR displays. As used herein, the term “a lower bit depth”refers to image data quantized in a coding space that is with the lowerbit depth; an example of lower bit depth comprises 8 bits, while theterm “a higher bit depth” refers to image data quantized in a codingspace that is with the higher bit depth; an example of higher bit depthis 12 bits or more. In particular, the term “a lower bit depth” or “ahigher bit depth” does not refer to least significant bits or mostsignificant bits of a pixel value.

In an example embodiment, the VDR image encoder (102) generates, basedon the image data in the base layer image container, a base layer videosignal, which may be outputted to a vide decoder (e.g., the VDR imagedecoder 150, or a first decoder 152-1 therein) in a downstream device.

In an example embodiment, a decoder (120) in the VDR image encoder (102)decodes the image data in the base layer image container into a decodedbase layer image in the 4:2:0 sampling format in the present example.The decoded base layer image is different from the 8 bit BL image (114),as the decoded base layer image comprises coding changes, roundingerrors and approximations introduced in the encoding and decodingoperations performed by the first encoder (116-1) and the decoder (120).

VDR image reconstruction data, in addition to what is contained in thebase layer video signal, may be delivered by the VDR image encoder to adownstream device in one or more enhancement layers separate from thebase layer. In some embodiments, the higher bit depth VDR image (112) inthe YCbCr color space may be predicted either from neighboring samplesin the same image frame (using intra prediction) or from samples frompast decoded image frames (inter prediction) that belong to the samelayer and are buffered as motion-compensated prediction referenceswithin a prediction image frame buffer. Inter-layer prediction may alsobe at least in part based on decoded information from other layers(e.g., the base layer).

In an example embodiment, the VDR image encoder (102) comprises aprediction processing unit (122) that performs one or more operationsrelating to prediction. Prediction as implemented by a predictionprocessing unit (e.g., 122) may reduce the overhead in reconstructing aVDR image by a VDR video decoder (e.g., 150 of FIG. 1B). In an exampleembodiment, the VDR image encoder (102) is configured to determine,based at least in part on the 12+ bit VDR image (112) and the decodedbase layer image, through intra or inter prediction (or estimation, orother methods), a set of mapping parameters for prediction (134). Theprediction processing unit (122) may generate a 12+ bit prediction imagein the YCbCr color space based on the set of mapping parameters (134)and the decoded base layer image. As used herein, examples of mappingparameters may include, but are limited only to, polynomial parametersused for prediction.

In an example embodiment, the VDR image encoder (102) is configured togenerate residual values (130) between the 12+ bit VDR image (112) andthe prediction image generated by the prediction processing unit (122).Residual values in a color channel (e.g., luminance channel) may bedifferences produced by subtraction operations (e.g., 126) in a linearor logarithmic domain. Alternatively and/or optionally, residual valuesin a color channel (e.g., luminance channel) may be ratios produced bydivision operations in a linear or logarithmic domain. In variousexample embodiments, one or more other mathematical representations andcorresponding operations may be used for the purpose of generatingresidual values (130) between the 12+ bit VDR image (112) and theprediction image.

In an embodiment, other than differences introduced by advancedquantization (or pseudo color grading process), the 12+ bit VDR image(112) and the 8 bit BL image (114) comprise the same image content. Inan embodiment, the 12+ bit VDR image (112) comprises the same chromainformation as the 8 bit BL image (114), other than quantization noisesor differences introduced by introduced by advanced quantization (orpseudo color grading process). In an embodiment, midtone and dark areasin the 12+ bit image (112) may be encoded in the base layer under theadvanced quantization, while highlight areas in the 12+ bit image (112)may be encoded in the enhancement layers under the same advancedquantization.

Additionally and/or optionally, no colorcorrection/alterations/distortion (e.g., clipping) is introduced intoonly base layer processing by the first encoding unit (116-1), thedecoding unit (120), or the prediction processing unit (122) in aprocessing path from the 8 bit BL image (114) to the prediction image.In an example embodiment, the prediction image comprises the same chromainformation as the 8 bit BL image (114), except for possible distortionsthat may be inherently present in the processing path (e.g., base layerdistortions caused by the base layer codec).

In an example embodiment, a non-linear quantizer (NLQ) 128 in the VDRimage encoder (102) is configured to quantize the residual values (130)in a 12+ bit digital representation to an 8 bit digital representation(or 8 bit residual values in the YCbCr color space) using one or moreNLQ parameters.

In an example embodiment, the VDR image encoder (102), or a secondencoder (116-2) therein, is configured to encode the 8 bit residualvalues in an enhancement layer image container, e.g., in the 4:2:0sampling format. The enhancement layer image container is logicallyseparate from the base layer image container in the base layer.

In an example embodiment, the VDR image encoder (102) generates, basedon the 8 bit residual values in the enhancement layer image container,an enhancement layer video signal, which may be outputted to a videodecoder (e.g., the VDR image decoder 150, or a second decoder 152-2therein).

In an example embodiment, the set of mapping parameters (134) and theNLQ parameters (132) may be transmitted to a downstream device (e.g.,the VDR image decoder 150) as a part of supplemental enhancementinformation (SEI) or other similar metadata carriages available in videobitstreams (e.g., in the enhancement layers).

One or more of the first encoder (116-1), the second encoder (116-2),and the decoder (120) (and 152-1, 152-2) may be implemented using one ormore of a plurality of codecs, such as H.264/AVC/HEVC, MPEG-2, VP8,VC-1, and/or others.

In an example embodiment, the VDR image decoder (150) is configured toreceive input video signals in multiple layers (or multiple bitstreams)comprising a base layer and one or more enhancement layers. As usedherein, the term “multi-layer” or “multiple layers” may refer to two ormore bitstreams that carry video or image signals having one or morelogical dependency relationships between one another (of the videosignals).

In an example embodiment, a first decoder (152-1) in the VDR imagedecoder (150) is configured to generate, based on a base layer videosignal, a decoded base layer image. In some embodiments, the firstdecoder (152-1) in the VDR image decoder (150) may be the same, orsubstantially similar to, the decoder (120) in the VDR image decoder(102). Likewise, the decoded base layer image in the VDR image decoder(150) and the decoded base layer image may be the same, or substantiallysimilar, provided that the decoded base layer images are sourced fromthe same VDR image (e.g., 106).

In an example embodiment, the VDR video decoder (150) comprises aprediction processing unit (158) that performs one or more operationsrelating to prediction. Prediction as implemented by a predictionprocessing unit may be used to efficiently reconstruct VDR images in aVDR video decoder (e.g., 150 of FIG. 1B). The prediction processing unit(158) is configured to receive the set of mapping parameters (134) andto generate, based at least in part on the set of mapping parameters(134) and the decoded base layer image, a 12+ bit prediction image.

In an example embodiment, a second decoder (152-2) in the VDR imagedecoder (150) is configured to retrieve, based on one or moreenhancement video signals, 8 bit residual values in an enhancement layerimage container.

In an example embodiment, a non-linear dequantizer (NLdQ) 154 in the VDRimage decoder (150) is configured to receive one or more NLQ parametersthrough the enhancement layers and to dequantize the 8 bit residualvalues to a 12+ bit digital representation (or 12+ bit residual valuesin the YCbCr color space) using the one or more NLQ parameters.

In an example embodiment, the VDR image decoder (150) is configured togenerate a reconstructed VDR image (160) based on the 12+ bit residualvalues (130) and the 12+ bit prediction image generated by theprediction processing unit (158). Reconstructed pixel values in a colorchannel (e.g., luminance channel) may be sums produced by additionoperations (e.g., 162) in a linear or logarithmic domain. Alternativelyand/or optionally, reconstructed values in a color channel (e.g.,luminance channel) may be products produced by multiplication operationsin a linear or logarithmic domain. In various example embodiments, oneor more other mathematical representations and corresponding operationsmay be used for the purpose of generating reconstructed pixel values(160) from the residual values and the prediction image.

2.2. Main Profile

FIG. 2A and FIG. 2B illustrate a VDR codec architecture in a mainprofile, in accordance with an example embodiment. As used herein, theterm main profile may refer to a profile that allows more complexitythan the baseline profile in a VDR coding system. For example, the mainprofile may allow operations in both the YCbCr or RGB color spaces andit may also allow operations in a variety of sub-sampling formats,including: 4:2:0, 4:2:2, and 4:4:4. In an example embodiment,predictions may be made in a RGB color space under a 4:4:4 samplingscheme; a polynomial/1D LUT prediction method, for example, may be usedfor prediction. In some embodiments, an upstream device that deliversVDR image data to downstream devices may comprise a VDR image encoder202 as illustrated in FIG. 2A, while a downstream device that receivesand processes the VDR image data may comprise a VDR image decoder 250.Each of the VDR image encoder 202 and the VDR image decoder 250 may beimplemented by one or more computing devices.

In an example embodiment, the VDR image encoder (202) is configured toreceive an input VDR image (206). The input VDR image (206) may be inany color space that supports a high dynamic range color gamut.

In an example embodiment, the input VDR image is a 12+ bit RGB image inan RGB color space, as illustrated in FIG. 2A. In an example, each pixelin the input VDR image comprises pixel values for red, green, and bluecolor channels defined in the RGB color space. Each pixel may optionallyand/or alternatively comprise upsampled or downsampled pixel values forone or more of the channels in the color space.

In an example embodiment, the VDR image encoder (202) is configured toperform advanced quantization on 12+ bit RGB image data in the VDR image206 (in a 4:4:4 sampling format in the present example) to generate 8bit RGB VDR data.

In an example embodiment, the VDR image encoder (202) is configured totransform the 8 bit RGB VDR data from a first color space (the RGB colorspace in the present example) to a second color space (e.g., an YCbCrcolor space). The color space transformation may be performed, forexample, by a RGB-2-YCbCr unit (208) in the VDR image encoder (202).

In an example embodiment, the VDR image encoder (202), or a downsampler(e.g., a 444-420 downsampler 210) therein, is configured to downsamplethe 8 bit VDR data in the YCbCr color space into an 8 bit downsampled BLimage 214 (e.g., in a 4:2:0 sampling format).

In an example embodiment, the VDR image encoder (202), or a firstencoder (216-1) therein, is configured to encode the 8 bit downsampledBL image (214) into image data in a base layer image container. In anexample embodiment, the image data in the base layer image container isnot optimized for viewing on SDR displays; rather, the image data in thebase layer image container is optimized to contain the maximal amount ofreconstructable information to represent the higher bit depth VDR imagedata in a lower bit depth image container and to minimize the amount ofVDR image reconstruction data (e.g., residual values 230) that needs tobe carried in the enhancement layers.

In an example embodiment, the VDR image encoder (202) generates, basedon the image data in the base layer image container, a base layer videosignal, which may be outputted to a video decoder (e.g., the VDR imagedecoder 250, or a first decoder 252-1 therein) in a downstream device.

In an example embodiment, a decoder (220) in the VDR image encoder (202)decodes the image data in the base layer image container into a decodedbase layer image in the 4:2:0 sampling format in the present example.The decoded base layer image is different from the 8 bit BL image (214),as the decoded base layer image comprise changes and errors, such asrounding errors and approximations, introduced in the encoding anddecoding operations performed by the first encoder (216-1) and thedecoder (220).

VDR image reconstruction data in addition to the base layer video signalmay be delivered by the VDR image encoder to a downstream device in oneor more enhancement layers separate from the base layer. The VDR image(206) in the RGB color space may be predicted either from neighboringsamples in the same image frame (using intra prediction) or from samplesfrom past decoded image frames (inter prediction) that belong to thesame layer and are buffered as motion-compensated prediction referenceswithin a prediction image frame buffer. Inter-layer prediction may alsobe at least in part based on decoded information from other layers(e.g., the base layer).

In an example embodiment, the VDR image encoder (202), or an upsampler(e.g., a 420-444 upsampler 212) therein, is configured to upsample thedecoded base layer image in the 4:2:0 sampling format into 8 bitupsampled image data (in the 4:4:4 sampling format in the presentexample).

In an example embodiment, the VDR image encoder (202), or an YCbCr-2-RGBunit (e.g., 236) therein, is configured to transform the 8 bit upsampledimage data from a non-prediction color space (the YCbCr color space inthe present example) to a prediction color space (e.g., the RGB colorspace).

In an example embodiment, the VDR image encoder (202) comprises aprediction processing unit (222) that performs one or more operationsrelating to prediction. Prediction as implemented by a predictionprocessing unit (e.g., 222) may reduce the overhead in reconstructing aVDR image by a VDR video decoder (e.g., 250 of FIG. 2B).

In an example embodiment, the VDR image encoder (202) is configured todetermine based at least in part on the 12+ bit VDR image (206) and theupsampled image data as transformed to the prediction color space,through intra or inter prediction (or estimation, or other methods), aset of mapping parameters (234) for prediction. The predictionprocessing unit (222) may generate, based on the set of mappingparameters (234) and the upsampled image data as transformed to theprediction color space, a 12+ bit prediction image in the RGB colorspace.

In an example embodiment, the VDR image encoder (202) is configured togenerate (RGB) residual values (230) between the 12+ bit VDR image (206)and the prediction image. Residual values in a color channel (e.g., Gchannel) may be differences produced by subtraction operations (e.g.,126) in a linear or logarithmic domain. Alternatively and/or optionally,residual values in a color channel (e.g., G channel) may be ratiosproduced by division operations in a linear or logarithmic domain. Invarious example embodiments, other mathematical representations andcorresponding operations/mappings/functions may be used for the purposeof generating residual values (230) between the 12+ bit VDR image (206)and the prediction image.

In an embodiment, the 12+ bit VDR image (206) comprises the same chromainformation as the 8 bit RGB VDR data, except for quantizationdifferences or noises introduced by the advanced quantization (or pseudocolor grading process). In an embodiment, midtone and dark areas in the12+ bit VDR image (206) may be encoded in the base layer under theadvanced quantization, while highlight areas in the 12+ bit VDR image(206) may be encoded in the enhancement layers under the same advancedquantization.

In an example embodiment, no extra colorcorrection/alterations/distortions (e.g., clipping) is introduced by theRGB-2-YCbCr unit (208), the downsampler (210), the first encoding unit(216-1), the decoding unit (220), the upsampler (212), the YCbCr-2-RGBunit (236), or the prediction processing unit (222) in a processing pathfrom the 8 bit RGB VDR data to the prediction image. In an exampleembodiment, the prediction image comprises the same chroma informationas the 8 bit RGB VDR data, except for possible distortions that may beinherently present in the processing path (e.g., base layer distortionscaused by the base layer codec, or errors from chroma reformatting indownsampling and upsampling).

In an example embodiment, a 444-to-420 downsampling and non-linearquantization unit (444-to-420& NLQ) 228 in the VDR image encoder (202)is configured to downsample and to quantize the residual values (230)from a 12+ bit digital representation in the 4:4:4 sampling format to an8 bit digital representation (or 8 bit RGB residual values) in the 4:2:0sampling format using one or more NLQ parameters.

In an example embodiment, the VDR image encoder (202), or a secondencoder (216-2) therein, is configured to encode the 8 bit residualvalues in an enhancement layer image container. The enhancement layerimage container is logically separate from the base layer imagecontainer.

In an example embodiment, the VDR image encoder (202) generates, basedon the 8 bit residual values in the enhancement layer image container,an enhancement layer video signal, which may be outputted to a videodecoder (e.g., the VDR image decoder 250, or a second decoder 252-2therein).

In an example embodiment, the set of mapping parameters (234) and theNLQ parameters (232) may be transmitted to a downstream device (e.g.,the VDR image decoder 250) as a part of supplemental enhancementinformation (SEI) or other similar metadata carriages available in videobitstreams (e.g., in the enhancement layers).

One or more of the first encoder (216-1), the second encoder (216-2),and the decoder (220) (252-1 and 252-2) may be implemented using one ormore of a plurality of codecs, such as H.264/AVC/HEVC, MPEG2, VP8, VC-1,and/or others.

In an example embodiment, the VDR image decoder (250) is configured toreceive input video signals in multiple layers (or multiple bitstreams)comprising a base layer and one or more enhancement layers.

In an example embodiment, a first decoder (252-1) in the VDR imagedecoder (250) is configured to generate, based on a base layer videosignal, a decoded (YCbCr) base layer image. In some embodiments, thefirst decoder (252-1) in the VDR image decoder (250) may be the same, orsubstantially similar to, the decoder (220) in the VDR image decoder(202). Likewise, the decoded base layer image in the VDR image decoder(250) and the decoded base layer image may be the same, or substantiallysimilar, provided that the decoded base layer images are sourced fromthe same VDR image (e.g., 206).

In an example embodiment, the VDR image decoder (250), or an upsampler(e.g., a 444-420 downsampler 266) therein, is configured to upsample thedecoded base layer image in a 4:2:0 sampling format into 8 bit upsampledimage data in the 4:4:4 sampling format in the present example.

In an example embodiment, the VDR image decoder (250), or an RGB-2-YCbCrunit (e.g., 264) therein, is configured to transform the 8 bit upsampledimage data from a non-prediction color space (the YCbCr color space inthe present example) to a prediction color space (e.g., the RGB colorspace).

In an example embodiment, the VDR video decoder (250) comprises aprediction processing unit (258) that performs one or more operationsrelating to prediction. Prediction as implemented by a predictionprocessing unit may be used to efficiently reconstruct VDR images in aVDR video decoder (e.g., 250 of FIG. 2B). The prediction processing unit(258) is configured to receive the set of mapping parameters (234) andto generate, based at least in part on the set of mapping parameters(234) and the 8 bit upsampled image data in the prediction color space,a 12+ bit prediction image.

In an example embodiment, a second decoder (252-2) in the VDR imagedecoder (250) is configured to retrieve, based on one or moreenhancement video signals, 8 bit (RGB) residual values in an enhancementlayer image container.

In an example embodiment, a non-linear dequantizer (NLdQ) 254 in the VDRimage decoder (250) and a 420-to-444 upsampler (268) are configured toreceive one or more NLQ parameters through the enhancement layers and todequantize and to upsample the 8 bit residual values in the 4:2:0sampling format to a 12+ bit digital representation (or 12+ bit residualvalues in the RGB color space) in the 4:4:4 sampling format using theone or more NLQ parameters.

In an example embodiment, the VDR image decoder (250) is configured togenerate a reconstructed VDR image (260) based on the 12+ bit residualvalues (230) and the 12+ bit prediction image generated by theprediction processing unit (258). Reconstructed pixel values in a colorchannel (e.g., G channel) may be sums produced by addition operations(e.g., 262) in a linear or logarithmic domain. Alternatively and/oroptionally, reconstructed values in a color channel (e.g., G channel)may be products produced by multiplication operations in a linear orlogarithmic domain. In various example embodiments, other mathematicalrepresentations and corresponding operations/functions/mappings may beused for the purpose of generating reconstructed pixel values (260) fromthe residual values and the prediction image.

Additionally and/or optionally, one or more of transform, quantization,entropy coding, image buffering, sample filtering, down-sampling,upsampling, interpolation, multiplexing, demultiplexing, interleaving,upscaling, downscaling, motion-compensating, disparity estimation,disparity compensation, depth estimation, depth compensation, encoding,decoding, etc., may be performed by a video encoder or decoder asdescribed herein.

3. ADVANCED QUANTIZATION

In some embodiments, advanced quantization such as performed by the VDRimage encoder (102 of FIG. 1A or 202 of FIG. 2A) is designed andimplemented to capture/preserve as many image details as possible in thebase layer. This minimizes the amount of residual values (e.g., 130 ofFIG. 1A or 230 of FIG. 2A) that needs to be encoded into an enhancementlayer video signal. Furthermore, the image details captured/preserved inthe base layer lend support in efficiently reconstructing VDR images bya downstream device such as a VDR image decoder (e.g., 150). Thepresence of accurate image details alleviates/reduces/removes visualartifacts that would otherwise be generated/amplified during lossycompression operations.

As discussed, unlike a base layer SDR image, optimized for SDR displays,generated by other techniques, a decoded base layer image undertechniques as described herein is not for viewing on SDR displays.Rather, a decoded base layer image under techniques as described hereinserves as intermediate image data for further generating residual valuesin a VDR image encoder and for further reconstructing higher bit depthVDR images in a VDR image decoder.

Under techniques as described herein, a color grading process designedfor producing best viewing experience on SDR displays is not needed andmay be disabled or avoided. External- or user-controlled colorcorrections that cause asymmetric (or different) clipping in theenhancement layer processing path and base layer processing path areavoided or disabled. Clipping levels in both enhancement layer and baselayer processing paths are fully controlled by a VDR image encoder undertechniques as described herein. Pixels that are color clipped in thebase layer image data may also be color clipped in the enhancement layerimage data.

Techniques as described herein may be used to reduce computationcomplexity for prediction including inter-layer prediction involving SDRimage data in the base layer and VDR image data in the enhancementlayers and are System-on-Chip (SoC) friendly. For example, a predictionprocess as described herein may be implemented as an inverse of advancedquantization (or pseudo color grading) as described herein. As theadvanced quantization may be fully controlled by a VDR image encoder asdescribed herein, the prediction process may also be fully controlled.In some embodiments, clipping levels and pixels with color clippings maybe fully controlled in the enhancement layer processing path and baselayer processing path so that a computationally efficient predictionmethod such as a first order polynomial mapping may be sufficient forgenerating and reconstructing prediction images.

In an example embodiment, higher bit depths (e.g., 16-bit) VDR data isdirectly quantized in advanced quantization (e.g., in FIG. 1A and FIG.2A) into lower bit depths (8-bit) base layer image data via a linearquantizer.

In some example embodiments, one or more of linear or non-linearquantizers may be used to quantize higher bit depth (e.g., 12+ bits)image data to lower bit depth (e.g., 8 bits) image data. Differentquantizers in different color spaces and/or in different color channelsmay be selected. For example, to alleviate/reduce/remove contouringartifacts (e.g., in smooth areas) and other artifacts, video signals maybe quantized in different color spaces and/or with different advancedquantization methods. In some embodiments, advanced quantization asdescribed herein may comprise one or more of linear quantization; linearstretching, curve-based/non-uniform quantization;probability-density-function (Pdf) optimized quantization (for example,LLoyd-Max quantization) based on histograms for a frame, multipleframes, a scene, multiple scenes, or one or more partitions within aframe, etc.; perceptual quantization; a vector quantization; anycombination of the foregoing (e.g., perceptual quantization followed byPdf-optimized quantization in a perceptual space). In some embodiments,a specific type of advanced quantization may have a correspondingrelationship with one or more types of prediction methods. For example,when uniform quantization is applied as advanced quantization, acorresponding type of prediction method used in prediction may be basedon a first order polynomial.

Quantization may be performed on an individual channel basis or on twoor more channels at the same time. In an example embodiment, vectorquantization may be performed across two or more color channels. Forexample, a coordinate system (e.g., 3D Cartesian) may be setup usingcolor channels in a color space as axes. Spatial transformation such asrotation may be performed in the coordinate system to create new axesthat are defined as combinations (or sums of projections) of the two ormore color channels in the color space. Pixel values in the two or morecolor channels as projected to form one of the new axes may be quantizedtogether by a quantizer over the one of the new axes.

In some embodiments, a specific advanced quantization method may beselected based on how well it can compress output multi-layer VDR imagedata while still maintaining high perceptual quality with the compressedoutput VDR image data on the VDR decoder side.

In some embodiments, a specific advanced quantization method may beselected to compensate weaknesses of codecs. For example, a codec maynot perform well in compressing black areas, and may even outputcontouring artifacts in a reconstructed VDR image. Advanced quantizationas described herein may use a specific curve (e.g., Sigmoid curve,mu-law, a human-perceptual based curve, etc.) to generate image datawith less contouring artifacts visible in a reconstructed VDR image.

A VDR encoder under techniques as described herein may take input VDRimage data as the only input for image content to be processed by theVDR encoder. While the input VDR image data may be provided toenhancement layer data processing, advanced quantization, which may beperformed on-the-fly (e.g., at the same wire speed at which the inputVDR is inputted into the VDR encoder), may be used to generate inputimage data to base layer data processing as described herein.

In some embodiments, an 8 bit quantization step (e.g., 128 of FIG. 1A or228 of FIG. 2A) as described herein may be preceded by a conversion tomake a video (e.g., VDR) signal look more like an SDR signal, asexisting encoders such as H.264 may have been adapted for processing anSDR signal. A variety of advanced quantization techniques that move thedynamic range of the VDR signal to look more like an SDR signal may beused. In an example embodiment, an invertible color grading method(e.g., Slope+Offset+Power+Hue+Saturation or SOP+HS) may be used totransform sparse data to targeted ranges. In another example embodiment,a tone mapping curve used in display management may be used to transformthe VDR signal to look more like an SDR signal. Here, the term “displaymanagement” refers to one or more operations that are performed to adapta VDR video signal to a dynamic range as supported by a specific displayor a specific range of displays.

Advanced quantization as described herein may be performed in one ormore different ways. Advanced quantization may perform a globalquantization in which an entire frame or an entire scene is quantizedusing a single setting. Advanced quantization may also perform apartition-based (local) quantization in which each frame is partitionedinto a plurality of non-overlapping regions and each non-overlappingregion is quantized using its own setting. Advanced quantization mayperform a partition-based (local) quantization in which each frame ispartitioned into a plurality of non-overlapping regions and eachnon-overlapping region is quantized using its own setting, but quantizersettings for a specific non-overlapping region are determined based onanalysis data derived from one or more overlapped regions. Advancedquantization may be applied in any of one or more different colorspaces. Examples of color spaces in which advanced quantization may beapplied include, but are not only limited to, any of: RGB color spaces,YCbCr color spaces, YCoCg color spaces, ACES color spaces, or othercolor spaces.

In some embodiments, a color space in which quantization is applied iskept the same as a color space in which prediction is performed. Thismay be so in both VDR image encoding process and VDR image decodingprocess. Color space transformation may be performed as appropriate if acolor space in which image rendering occurs is different from a colorspace in which quantization occurs.

4. LINEAR STRETCHING

In an example embodiment, a scene-adaptive dynamic range adjustmentquantization method may be applied in advanced quantization, asillustrated in FIG. 1A and FIG. 2A, in an YCbCr color space, asillustrated in FIG. 3, or an RGB color space. The maximal value in colorchannel i within one considered scene may be denoted as v_(i,max). Theminimal value in color channel i within one considered scene may bedenoted as v_(i,min). The range as defined by minimal and maximal and/ordistribution of data points within the range may be changed based onimage content from frame to frame, from multiple frames to multipleframes, from scene to scene, from multiple scene to multiple scene, fromprogram to program, etc.

A to-be-processed pixel value in color channel i may be denoted asv_(i). The following expression may be held true where a VDR (e.g.,luminance) coding space is in 16 bits (or 12+ bits of FIG. 1 and FIG.2):0≦v _(i,min) ≦v _(i,max)≦2¹⁶−1  (1)

The scene-adaptive dynamic range adjustment quantization method maps theentire range [v_(i,min), v_(i,max)] to an 8-bit YCbCr 709 standard range[s_(i,min), s_(i,max)], as follows:

$\begin{matrix}{{s_{i} = {{round}( {{\frac{s_{i,\max} - s_{i,\min}}{v_{i,\max} - v_{i,\min}} \cdot ( {v_{i} - v_{i,\min}} )} + v_{i,\min}} )}},} & (2)\end{matrix}$where s_(i) denotes the converted pixel value in the image datagenerated by the advanced quantization, as illustrated in FIG. 1A andFIG. 2A. In expression (2) the round( ) operation guarantees that theoutput will be an integer. Rounding may also be followed by a clippingfunction. For example, negative values may be clipped to zero andpositive values larger than 255 may be clipped to 255.

As illustrated in FIG. 3, the scene-adaptive dynamic range adjustmentquantization may be used to fully utilize the whole 8 bit dynamic range.The horizontal axis of the quantization-range-versus-frame-index chartin FIG. 3 represents a frame index variable. The minimum value forlinear stretching, s_(i,min) as indicated by plot 302, in each frame maybe set the same as the minimum value, v_(i,min) as indicated by plot304, in the frame. The maximum value for linear stretching, s_(i,max) asindicated by plot 306, in each frame, however, may be set to be no lessthan the maximum value, v_(i,max) as indicated by plot 308, in theframe. As depicted in FIG. 3, in frame 2200, under other codingtechniques (e.g., other than linear stretching coding techniques), themaximum value is about 140. In contrast, using the linear stretchingtechniques as described herein, the maximum value for frame 2200 isextended to about 225. Thus, linear stretching as described hereinprovides more quantization steps relative to the other coding techniquesand hence provides better resolution details. As illustrated, clippingstarts occurring at a frame near frame 2400 and continues to frame 2600for both linear stretching and the other techniques.

5. EXAMPLE PROCESS FLOWS

FIG. 4A illustrates an example process flow according to an exampleembodiment of the present invention. In some example embodiments, one ormore computing devices or components may perform this process flow. Inblock 402, a multi-layer VDR video encoder (e.g., 102 of FIG. 1)receives an input visual dynamic range (VDR) image in a sequence ofinput images.

In block 404, the multi-layer VDR video encoder (102) selects a specificadvanced quantization method from one or more available advancedquantization methods.

In block 406, the multi-layer VDR video encoder (102) applies thespecific advanced quantization method to the input VDR image to generatean input base layer image. In an example embodiment, the input VDR imagecomprises higher bit depth VDR image data, whereas the input base layerimage comprises lower bit depth VDR image data.

In block 408, the multi-layer VDR video encoder (102) compresses imagedata derived from the input base layer image into a base layer (BL)video signal.

In block 410, the multi-layer VDR video encoder (102) compresses atleast a portion of image data derived from the input VDR image into oneor more enhancement layer (EL) video signals.

In an example embodiment, the multi-layer VDR video encoder (102)decodes a base layer image from the BL video signal, the base layerimage corresponding to the input base layer image; selects a predictionmethod from one or more prediction methods; generates a prediction imagebased at least in part on the base layer image using the predictionmethod; generates residual values based on the prediction image and theinput VDR image; applies non-linear quantization to the residual valuesto generate output EL image data, the residual values comprising higherbit depth values, and the output EL image data comprising lower bitdepth values; and compresses the output EL image data into the one ormore EL video signals.

In an example embodiment, the prediction method is selected based on acorrespondence relationship between the advanced quantization method andthe prediction method.

In an example embodiment, the advanced quantization method comprises oneor more of global quantization, linear quantization, linear stretching,curve-based quantization, probability-density-function (Pdf) optimizedquantization, LLoyd-Max quantization, partition-based quantization,perceptual quantization, vector quantization, or other types ofquantization.

In an example embodiment, the sequence of input images comprises asecond different VDR input image; and the multi-layer video encoder(102) selects a second different specific advanced quantization methodfrom the one or more available advanced quantization methods; appliesthe second specific advanced quantization method to the second input VDRimage to generate a second input base layer image; compresses secondimage data derived from the second input base layer image into the baselayer (BL) video signal; and compresses at least a portion of image dataderived from the second input VDR image into the one or more enhancementlayer (EL) video signals.

In an example embodiment, the multi-layer video encoder (102) decodes asecond different BL image from the base layer video signal, the secondBL image corresponding to the second input BL image; selects a seconddifferent prediction method from the one or more prediction methods;generates a second prediction image based at least in part on the secondBL image using the second prediction method; computes second differentresidual values based on the second prediction image and the secondinput VDR image; applies non-linear quantization to the second residualvalues to generate second output EL image data, the second residualvalues comprising higher bit depth values, and the second output ELimage data comprising lower bit depth values; and compresses the outputEL image data into the one or more EL video signals.

In an example embodiment, the image data in the input base layer imageis compressed by a first 8 bit encoder in a VDR encoder into the BLvideo signal, whereas the at least a portion of image data in the inputVDR image is compressed by a second 8 bit encoder in the VDR encoderinto the one or more enhancement layer (EL) video signals.

In an example embodiment, the advanced quantization method is selectedbased on one or more factors including but not limited to minimizing anamount of image data to be encoded into the one or more EL video signalsrelative to the input VDR image.

In an example embodiment, the advanced quantization method is selectedbased on one or more factors including but not limited to any of one ormore characteristics determined from the input VDR image.

In an example embodiment, color grading by a colorist is disabled afterthe input VDR image is received by the multi-layer video encoder (102).

In an example embodiment, a first image container is used to hold theimage data derived from the input base layer image, whereas a seconddifferent image container is used to hold the at least a portion ofimage data in the input VDR image. In an example embodiment, at leastone of the first image container and the second image containercomprises pixel values in one or more channels in a color space. In anexample embodiment, at least one of the first image container and thesecond image container is selected from a plurality of image containersassociated with a plurality of sampling schemes, and wherein theplurality of sampling schemes comprises any of: a 4:4:4 sampling scheme,a 4:2:2 sampling scheme, a 4:2:0 sampling scheme, or other samplingschemes.

In an example embodiment, the multi-layer video encoder (102) convertsone or more input VDR images represented, received, transmitted, orstored with one or more input video signals into one or more output VDRimages represented, received, transmitted, or stored with one or moreoutput video signals.

In an example embodiment, at least one of the input VDR image and theone or more EL video signals comprises image data encoded in one of ahigh dynamic range (HDR) image format, a RGB color space associated withthe Academy Color Encoding Specification (ACES) standard of the Academyof Motion Picture Arts and Sciences (AMPAS), a P3 color space standardof the Digital Cinema Initiative, a Reference Input MediumMetric/Reference Output Medium Metric (RIMM/ROMM) standard, an sRGBcolor space, or a RGB color space associated with the BT.709Recommendation standard of the International Telecommunications Union(ITU).

FIG. 4B illustrates an example process flow according to an exampleembodiment of the present invention. In some example embodiments, one ormore computing devices or components may perform this process flow. Inblock 452, a multi-layer video decoder (e.g., 150 of FIG. 1B) generatesat least a portion of image data of a VDR image, in a sequence of inputimages, based on one or more enhancement layer (EL) video signals.

In block 454, the multi-layer video decoder (150) generates a base layerimage based on a base layer (BL) video signal, the base layer imagecomprising lower bit depth VDR image data, of the VDR image, generatedby a specific advanced quantization method selected from one or moreavailable advanced quantization methods.

In block 456, the multi-layer video decoder (150) reconstructs a higherbit depth version of the VDR image based on the base layer image and theat least a portion of image data.

In an example embodiment, the multi-layer video decoder (150) receivesprediction metadata including, but not limited only to, a set of mappingparameters; determines a prediction method based on the predictionmetadata; generates a prediction image based at least in part on thebase layer image using the prediction method; reconstructs the higherbit depth version of the VDR image by combining the prediction imagewith the at least a portion of image data derived from the one or moreEL video signals.

In an example embodiment, the prediction method corresponds to theadvanced quantization method.

In an example embodiment, the advanced quantization method comprises oneor more of global quantization, linear quantization, linear stretching,curve-based quantization, probability-density-function (Pdf) optimizedquantization, LLoyd-Max quantization, partition-based quantization,perceptual quantization, vector quantization, or other types ofquantization.

In an example embodiment, the base layer image is derived by a first 8bit decoder in a VDR decoder from the BL video signal, and wherein theat least a portion of image data in the VDR image is derived by a second8 bit decoder in the VDR decoder from the one or more enhancement layer(EL) video signals.

In an example embodiment, the advanced quantization method was selectedbased on one or more factors including, but not limited to, minimizingan amount of image data to be derived from the one or more EL videosignals relative to a source VDR image.

In an example embodiment, a first image container is used to hold theimage data in the base layer image, whereas a second different imagecontainer is used to hold the at least a portion of image data of theVDR image. In an example embodiment, at least one of the first imagecontainer and the second image container comprises pixel values in oneor more channels in a color space. In an example embodiment, at leastone of the first image container and the second image container isselected from a plurality of image containers associated with aplurality of sampling schemes, and wherein the plurality of samplingschemes comprises any of: a 4:4:4 sampling scheme, a 4:2:2 samplingscheme, a 4:2:0 sampling scheme, or other sampling schemes.

In an example embodiment, the multi-layer video decoder (150) processesone or more VDR images represented, received, transmitted, or storedwith one or more input video signals.

In an example embodiment, at least a portion of the higher bit depthversion of the VDR image comprises image data encoded in one of a highdynamic range (HDR) image format, a RGB color spaces associated with theAcademy Color Encoding Specification (ACES) standard of the Academy ofMotion Picture Arts and Sciences (AMPAS), a P3 color space standard ofthe Digital Cinema Initiative, a Reference Input Medium Metric/ReferenceOutput Medium Metric (RIMM/ROMM) standard, an sRGB color space, or a RGBcolor space associated with the BT.709 Recommendation standard of theInternational Telecommunications Union (ITU).

In various example embodiments, an encoder, a decoder, a system, anapparatus, or one or more other computing devices performs any or a partof the foregoing methods as described.

6. ADAPTIVE DYNAMIC RANGE ADJUSTMENT

Fade-ins and fade-outs are special scene-transition effects that arecommonly used in video production. In a fade-in, brightness increasesgradually until the scene is at full brightness. During a fade-out, ascene starts at full brightness and disappears gradually. Because of thechange in luminance during these transitions, motion estimationtechniques may fail to accurately determine the best motion vectors,resulting in larger residuals and more inefficient video coding.

In certain embodiments where the linear stretching quantizer is applied(e.g., equation (2)), it is desirable to maintain a relatively constantVDR to base layer (BL) quantization step within a scene. This approach,denoted herein as “scene-based adaptation”, reduces the amount ofquantization-related metadata that needs to be transmitted from theencoder to the decoder and also maintains a relatively constantbrightness in a scene, which assists the subsequent compression process.However, such an approach may not be suitable during fade-ins orfade-outs. As described herein, a “frame-by-frame based adaptation” maybe better suited for such transitions.

Suppose there are F frames during a fade-in or fade-out transition. Fora certain color component (e.g., Luminance Y), for the i-th frame in theoriginal VDR sequence, denote as v_(H,i) and v_(L,i) (i=0, . . . , F−1)as the maximum and minimum values for that color component,respectively. Similarly, denote as c_(H,i) and c_(L,i) (i=0, . . . ,F−1) as the maximum and minimum value for the corresponding colorcomponent in the i-th BL frame, respectively. Using the linearstretching quantization method, from equation (2), the value of the j-thpixel in the i-th frame of the quantized base layer stream may beexpressed as:

$\begin{matrix}{{s_{ji} = {{Q_{i}( v_{ji} )} = \lfloor {{\frac{c_{H,i} - c_{L,i}}{v_{H,i} - v_{L,i}}( {v_{ji} - v_{L,i}} )} + c_{L,i} + O} \rfloor}},} & (3)\end{matrix}$

where v_(ji) denotes the value of the j-th pixel in the i-th VDR frameand O is a rounding offset (e.g., O=0.5 or O=0). As applied herein, thefloor function └x┘ computes the greatest integer less than or equal tox.

For a fade-out scene, the first frame should have the maximal dynamicrange, namely, v_(H,0)≧v_(H,i) for 0<i<F.

For a fade-in scene, the last frame should have the maximal dynamicrange, namely, v_(H,F−1)≧v_(H,i) for 0≦i<F−1.

Given the above formulation, a problem that arises is how in equation(3) one may adaptively adjust the {c_(H,i)|i=0, . . . , F−1} and{c_(L,i)|i=0, . . . , F−1} parameters in order to optimize subsequentcoding performance.

Full-Search Method

In one embodiment one may try all possible combinations of {c_(H,i)|i=0,. . . , F−1} and {c_(L,i)|i=0, . . . , F−1} and select those variablesthat provide the best overall compression. However, even if one setsc_(L,i)=0, for 8-bit data, there are 255^(F) possible combinations forc_(H,i), which may be impractical to try and test in real-time encoding.

The Equal Max-Value Method

In another embodiment, one may set all c_(H,i) values (i=0, . . . , F−1)to a scene-dependent maximal value, c_(H,max). In an embodiment,c_(H,max) may represent the value being used in either the previous ornext scene with constant brightness, namely, a scene with no fade-in orfade-out (e.g., c_(H,i)=c_(H,max)=255, for all i in [0, F−1]).Similarly, c_(L,i) may be set to the minimal value, c_(L,min), which wasused in the previous or next scene without fade in/fade out (e.g.,c_(L,I)=c_(L,min)=0, for all i in [0, F−1].) In such an embodiment, allBL frames within the fade-in or fade-out scene will have the samedynamic range [c_(L,min) c_(H,max)]; however, the VDR to BL quantizationstep from frame to frame may be different. From equation (3), thisadaptive quantization approach (also to be referred asframe-by-frame-adaptation) for fade-in and fade-out transitions may beexpressed as:

$\begin{matrix}{s_{ji} = {{Q_{i}( v_{ji} )} = {\lfloor {{\frac{c_{H,\max} - c_{L,\min}}{v_{H,i} - v_{L,i}}( {v_{ji} - v_{L,i}} )} + c_{L,\min} + O} \rfloor.}}} & (4)\end{matrix}$

A decision algorithm to detect whether to apply scene-based adaptation(e.g., apply equations (2) or (3) with constant quantization for thewhole scene) or frame-by-frame adaptation (e.g., apply equation (4)) isdescribed next.

Decision Algorithm

In an embodiment, consider two consecutive VDR frames, say framesv_(i−1) and v_(i). Then, a decision algorithm may be derived bycomparing histograms of the corresponding quantized BL frames s_(i−1)and s_(i). While the algorithm is described for a single color component(e.g., luminance), the operations may be repeated for all colorcomponent.

Step 1: Assume frame-by-frame (fbf) adaptive quantization and compute BLpixel values

Given frames and v_(i−1) and v_(i), one may apply equation (4) tocompute pixel values in the corresponding BL frames as:

(a) For frame i−1

$\begin{matrix}{s_{{ji} - 1}^{fbf} = {{Q_{i - 1}^{fbf}( v_{{ji} - 1} )} = {\lfloor {{\frac{c_{H,\max} - c_{L,\min}}{v_{H,i} - v_{L,i}}( {v_{ji} - v_{L,i}} )} + c_{L,\min} + O} \rfloor.}}} & (5)\end{matrix}$

(b) For frame i

$\begin{matrix}{s_{ji}^{fbf} = {{Q_{i}^{fbf}( v_{ji} )} = {\lfloor {{\frac{c_{H,\max} - c_{L,\min}}{v_{H,i} - v_{L,i}}( {v_{ji} - v_{L,i}} )} + c_{L,\min} + O} \rfloor.}}} & (6)\end{matrix}$

Without loss of generality, assuming 8-bits per color component in theBL stream, for frames and s_(i−1) one may use the output of equations(5) and (6) to compute the corresponding histograms, each with 256 bins,as H_(i−1) ^(fbf)(n) and H_(i) ^(fbf)(n), for n=0, 1, . . . , 255. Asused herein, the term histogram denotes a function that counts thenumber of observed pixels that fall into each one of the possibledistinct pixel values. For example, H_(i) ^(fbf)(20)=10, denotes that 10pixels in frame i−1 have the value 20.

Step 2: Calculate the mean-square difference between H_(i−1) ^(fbf)(n)and H_(i) ^(fbf)(n)

Given the histograms computed in Step 1, one may compute theirmean-square difference as

$\begin{matrix}{D_{i}^{fbf} = {\frac{1}{256}{\sum\limits_{n = 0}^{255}{{{{H_{i - 1}^{fbf}(n)} - {H_{i}^{fbf}(n)}}}^{2}.}}}} & (7)\end{matrix}$

The process may now be repeated under the assumption of using ascene-based adaptive (sb) quantization.

Step 3: Calculate the minimum and maximum pixel values among frame i−1and frame iv _(Lmin)=min{v _(L,i−1) ,v _(L,i)},andv _(Hmax)=max{v _(H,i−1) ,v _(H,i)}.

Then, given frames v_(i−1) and v_(i), one may apply those values andequation (3) to compute the corresponding BL pixel values as

$\begin{matrix}{{s_{{ji} - 1}^{sb} = {{Q_{i - 1}^{sb}( v_{{ji} - 1} )} = \lfloor {{\frac{c_{H,\max} - c_{L,\min}}{v_{H\max} - v_{L\min}}( {v_{{ji} - 1} - v_{L\min}} )} + c_{L,\min} + O} \rfloor}},\mspace{85mu}{and}} & (8) \\{\mspace{85mu}{s_{ji}^{sb} = {{Q_{i}^{sb}( v_{ji} )} = {\lfloor {{\frac{c_{H,\max} - c_{L,\min}}{v_{H\max} - v_{L\min}}( {v_{ji} - v_{Lmin}} )} + c_{L,\min} + O} \rfloor.}}}} & (9)\end{matrix}$

Using the output of equations (8) and (9), one may compute framehistograms H_(i) ^(sb)(n), and H_(i−1) ^(sb)(n), for n=0, 1, . . . ,255.

Step 4: Calculate the mean-square difference between H_(i−1) ^(sb)(n)and H_(i) ^(sb)(n)

$\begin{matrix}{D_{i}^{sb} = {\frac{1}{256}{\sum\limits_{n = 0}^{255}{{{{H_{i - 1}^{sb}(n)} - {H_{i}^{sb}(n)}}}^{2}.}}}} & (10)\end{matrix}$Step 5: An adaptive decision to apply either frame-by-frame orscene-based adaptation may be based on the difference between the twomean-square differences:

  if D_(i) ^(fbf) < D_(i) ^(sb) use frame-by-frame adjustment else usescene-based adjustment.FIG. 6 summarizes an embodiment of the decision algorithm as describedherein. In step 610, the process accesses two consecutive images (orframes) in a sequence of input VDR images. Using the methods describedherein, steps 625 and 630 compute two alternative representations of thecorresponding BL images. Step 625 computes the BL frames usingframe-by-frame adaptation (e.g., using equations (5) and (6). Step 630computes the BL images using scene-based adaptation (e.g., usingequations (8) and (9)). Based on these computed BL images, steps 625 and630 may compute the corresponding histograms (e.g., H_(i−1) ^(fbf)(n),H_(i) ^(fbf)(n), H_(i) ^(sb)(n), and H_(i−1) ^(sb)(n)). Given thesehistograms, for each set of histograms, steps 635 and 640 may compute afirst and a second mean-square difference (e.g., D_(i) ^(fbf) inequation (7) and D_(i) ^(sb) in equation (10)). Finally, in step 650,one may compare the two mean-square differences and select as thequantization method the method that yields the histograms with thesmallest mean square difference.

7. IMPLEMENTATION MECHANISMS—HARDWARE OVERVIEW

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 5 is a block diagram that illustrates a computersystem 500 upon which an example embodiment of the invention may beimplemented. Computer system 500 includes a bus 502 or othercommunication mechanism for communicating information, and a hardwareprocessor 504 coupled with bus 502 for processing information. Hardwareprocessor 504 may be, for example, a general purpose microprocessor.

Computer system 500 also includes a main memory 506, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 502for storing information and instructions to be executed by processor504. Main memory 506 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 504. Such instructions, when stored innon-transitory storage media accessible to processor 504, rendercomputer system 500 into a special-purpose machine that is customized toperform the operations specified in the instructions.

Computer system 500 further includes a read only memory (ROM) 508 orother static storage device coupled to bus 502 for storing staticinformation and instructions for processor 504. A storage device 510,such as a magnetic disk or optical disk, is provided and coupled to bus502 for storing information and instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa liquid crystal display, for displaying information to a computer user.An input device 514, including alphanumeric and other keys, is coupledto bus 502 for communicating information and command selections toprocessor 504. Another type of user input device is cursor control 516,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 504 and forcontrolling cursor movement on display 512. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

Computer system 500 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 500 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 500 in response to processor 504 executing one or more sequencesof one or more instructions contained in main memory 506. Suchinstructions may be read into main memory 506 from another storagemedium, such as storage device 510. Execution of the sequences ofinstructions contained in main memory 506 causes processor 504 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 510.Volatile media includes dynamic memory, such as main memory 506. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 502. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 504 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 502. Bus 502 carries the data tomain memory 506, from which processor 504 retrieves and executes theinstructions. The instructions received by main memory 506 mayoptionally be stored on storage device 510 either before or afterexecution by processor 504.

Computer system 500 also includes a communication interface 518 coupledto bus 502. Communication interface 518 provides a two-way datacommunication coupling to a network link 520 that is connected to alocal network 522. For example, communication interface 518 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 518 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 518sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 520 typically provides data communication through one ormore networks to other data devices. For example, network link 520 mayprovide a connection through local network 522 to a host computer 524 orto data equipment operated by an Internet Service Provider (ISP) 526.ISP 526 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 528. Local network 522 and Internet 528 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 520and through communication interface 518, which carry the digital data toand from computer system 500, are example forms of transmission media.

Computer system 500 can send messages and receive data, includingprogram code, through the network(s), network link 520 and communicationinterface 518. In the Internet example, a server 530 might transmit arequested code for an application program through Internet 528, ISP 526,local network 522 and communication interface 518.

The received code may be executed by processor 504 as it is received,and/or stored in storage device 510, or other non-volatile storage forlater execution.

8. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

In the foregoing specification, example embodiments of the inventionhave been described with reference to numerous specific details that mayvary from implementation to implementation. Thus, the sole and exclusiveindicator of what is the invention, and is intended by the applicants tobe the invention, is the set of claims that issue from this application,in the specific form in which such claims issue, including anysubsequent correction. Any definitions expressly set forth herein forterms contained in such claims shall govern the meaning of such terms asused in the claims. Hence, no limitation, element, property, feature,advantage or

attribute that is not expressly recited in a claim should limit thescope of such claim in any way. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense.

The invention claimed is:
 1. A method, comprising: receiving an inputvisual dynamic range (VDR) image in a sequence of input images, whereinthe input VDR image is of a first bit depth; selecting a specificadvanced quantization function from a plurality of available advancedquantization functions for mapping VDR image data of the first bit depthto base layer image data of a second bit depth lower than the first bitdepth; applying the specific advanced quantization function to the inputVDR image to generate an input base layer image, wherein the input baselayer image is of the second bit depth; wherein the input base layerimage as generated by the selected specific advanced quantizationfunction is non-backward-compatible; compressing image data derived fromthe input base layer image into a base layer (BL) video signal; andcompressing at least a portion of image data derived from the input VDRimage into one or more enhancement layer (EL) video signals, wherein theadvanced quantization method is selected from the plurality of availableadvanced quantization functions based on one or more factors includingminimizing an amount of image data to be encoded into the one or more ELvideo signals relative to the input VDR image.
 2. The method as recitedin claim 1, further comprising: decoding a BL image from the BL videosignal, the BL image corresponding to the input BL image; selecting aprediction method from one or more prediction methods; generating aprediction image based at least in part on the BL image using theprediction method; generating residual values based on the predictionimage and the input VDR image; applying non-linear quantization to theresidual values to generate output EL image data, the residual valuescomprising higher bit depth values, and the output EL image datacomprising lower bit depth values; and compressing the output EL imagedata into the one or more EL video signals.
 3. The method as recited inclaim 2, wherein the prediction method is selected based on acorrespondence relationship between the advanced quantization method andthe prediction method.
 4. The method as recited in claim 1, wherein theadvanced quantization method comprises one or more of globalquantization, linear quantization, linear stretching, curve-basedquantization, probability-density-function (Pdf) optimized quantization,LLoyd-Max quantization, partition-based quantization, perceptualquantization, or cross-color channel/vector quantization.
 5. The methodas recited in claim 1, wherein the sequence of input images comprises asecond different VDR input image; and the method further comprising:selecting a second different specific advanced quantization method fromthe plurality of available advanced quantization methods; applying thesecond specific advanced quantization method to the second input VDRimage to generate a second input BL image; compressing second image dataderived from the second input base layer image into the BL video signal;and compressing at least a portion of image data derived from the secondinput VDR image into the one or more EL video signals.
 6. The method asrecited in claim 5, further comprising: decoding a second different BLimage from the BL video signal, the second BL image corresponding to thesecond input BL image; selecting a second different prediction methodfrom the one or more prediction methods; generating a second predictionimage based at least in part on the second BL image using the secondprediction method; generating second different residual values based onthe second prediction image and the second input VDR image; applyingnon-linear quantization to the second residual values to generate secondoutput EL image data, the second residual values comprising higher bitdepth values, and the second output EL image data comprising lower bitdepth values; and compressing the output EL image data into the one ormore EL video signals.
 7. The method as recited in claim 1, wherein theimage data in the input BL image is compressed by a first 8 bit encoderin a VDR encoder into the BL video signal, and wherein the at least aportion of image data in the input VDR image is compressed by a second 8bit encoder in the VDR encoder into the one or more EL video signals. 8.The method as recited in claim 1, wherein a first image container isused to hold the image data derived from the input BL image, and whereina second different image container is used to hold the at least aportion of image data in the input VDR image.
 9. The method as recitedin claim 8, wherein at least one of the first image container and thesecond image container comprises pixel values in one or more channels ina color space.
 10. The method as recited in claim 8, wherein at leastone of the first image container and the second image container isselected from a plurality of image containers associated with aplurality of sampling schemes, and wherein the plurality of samplingschemes comprises any of: a 4:4:4 sampling scheme, a 4:2:2 samplingscheme, or a 4:2:0 sampling scheme.
 11. The method as recited in claim1, further comprising: determining a specific profile for processing theinput VDR image into the BL and EL video signals; and performing one ormore operations related to the specific profile in processing the inputVDR image into the BL and EL video signals.
 12. A method, comprising:generating at least a portion of image data of a VDR image of a firstbit depth, in a sequence of input images, based on one or moreenhancement layer (EL) video signals; generating a base layer (BL)image, of a second bit depth lower than the first bit depth, based on aBL video signal, the BL image was generated by applying a specificadvanced quantization method to the VDR image, the specific advancedquantization method is selected from a plurality of available advancedquantization methods for mapping VDR image data of the first bit depthto base layer image data of the second bit depth; wherein the BL imageas generated by the selected specific advanced quantization function isnon-backward-compatible; and reconstructing a version of the VDR imagebased on the BL image and the at least a portion of image data of theVDR image, the version of the VDR image comprises reconstructed VDRimage data of the first bit depth, wherein the advanced quantizationmethod was selected from the plurality of available advancedquantization methods based on one or more factors including minimizingan amount of image data to be derived from the one or more EL videosignals relative to a source VDR image.
 13. The method as recited inclaim 12, further comprising: receiving prediction metadata including aset of mapping parameters; determining a prediction method based on theprediction metadata; generating a prediction image based at least inpart on the BL image using the prediction method; reconstructing thehigher bit depth version of the VDR image by combining the predictionimage with the at least a portion of image data derived from the one ormore EL video signals.
 14. The method as recited in claim 13, whereinthe prediction method corresponds to the advanced quantization method.15. The method as recited in claim 13, wherein the advanced quantizationmethod comprises one or more of: global quantization, linearquantization, linear stretching, curve-based quantization,probability-density-function (Pdf) optimized quantization, LLoyd-Maxquantization, partition-based quantization, perceptual quantization, orvector quantization.
 16. The method as recited in claim 13, wherein theBL image is derived by a first 8 bit decoder in a VDR decoder from theBL video signal, and wherein the at least a portion of image data in theVDR image is derived by a second 8 bit decoder in the VDR decoder fromthe one or more enhancement layer (EL) video signals.
 17. The method asrecited in claim 12, wherein a first image container is used to hold theimage data in the BL image, and wherein a second different imagecontainer is used to hold the at least a portion of image data of theVDR image.
 18. The method as recited in claim 17, wherein at least oneof the first image container and the second image container comprisespixel values in one or more channels in a color space.
 19. The method asrecited in claim 17, wherein at least one of the first image containerand the second image container is selected from a plurality of imagecontainers associated with a plurality of sampling schemes, and whereinthe plurality of sampling schemes comprises at least a 4:4:4 samplingscheme, a 4:2:2 sampling scheme, a 4:2:0 sampling scheme, or othersampling schemes.
 20. The method as recited in claim 12, furthercomprising: determining a specific profile relating to the BL and ELvideo signals; and performing one or more operations related to thespecific profile in reconstructing the higher bit depth version of theVDR image from the BL and EL video signals.
 21. A non-transitorycomputer readable storage medium, storing computer instructions, whichwhen executed by one or more processors cause performing the method asrecited in claim
 1. 22. A non-transitory computer readable storagemedium, storing computer instructions, which when executed by one ormore processors cause performing the method as recited in claim
 12. 23.An apparatus comprising: one or more processors; a non-transitorycomputer readable storage medium, storing computer instructions, whichwhen executed by the one or more processors cause performing the methodas recited in claim
 1. 24. An apparatus comprising: one or moreprocessors; a non-transitory computer readable storage medium, storingcomputer instructions, which when executed by the one or more processorscause performing the method as recited in claim 12.